Audio Information Extraction from Arbitrary Sound Recordings

Numerous archives of entertainment soundtracks and other recordings such as environmental noise samples have imposed a big data challenge in audio related industries. This necessitates the use of machine audition and retrieval tools to extract semantic information for various applications. Speech recognition, environmental noise classification and music information retrieval tools haven been developed in the past for specific purposes. Combined use of these tools to process arbitrary sound recordings remains challenging: overlap of diverse sources mitigates the classification, resulting in poor recognition and/or missing content. Following a review of a universal framework for arbitrary soundtrack information mining proposed by the authors, a new solution to the overlapped sound sources has been developed in this paper by iterative signal cleaning techniques. The system classifies the arbitrary audio signals into music, speech, ambient sounds and silence, allowing overlap. Validation tests have shown that the new techniques can reduce or eliminate information losses in machine audition, hence improving the usability of machine audition in processing real-world audio archives. This paper will also discuss the dataset and principles, present the validation results and discuss potential applications.